Skip to content

Time dilation methods to suppress CFL#946

Open
Maxwell-Rosen wants to merge 59 commits intomainfrom
time-dilation-reorg
Open

Time dilation methods to suppress CFL#946
Maxwell-Rosen wants to merge 59 commits intomainfrom
time-dilation-reorg

Conversation

@Maxwell-Rosen
Copy link
Copy Markdown
Collaborator

@Maxwell-Rosen Maxwell-Rosen commented Feb 10, 2026

Feature

Summary

Purpose: This PR adds methods to dilate time in extreme regions of phase space to control CFL.

Issue link: Based on #894, fixes #895.

Implementation Details

Math basis: Time_dilation_notes (1).pdf
Essentially, we multiply fdot by a small factor beta, determined by the CFL condition.

Key changes: The time dilation is contained inside the fdot multiplier. Parameters are added to the time_rate_multiplier struct in the input file, which contains everything relevant for time dilation.

New operations are added to array ops. gkyl_array_min_by_cell, gkyl_array_min_by_cell_range, gkyl_array_invert_by_cell with appropriate unit tests. These are neccisary to compute

$$ {\rm min} (1, \Omega_{max} / \Omega_{cfl} ) $$

Utilizes the masks introduced in #889.

Dependencies: Mask, array ops, fdot_multiplier

Automated testing: This feature is added to multiple mirror regression tests, which include every combination of time dilation configurations

Example Use

  struct gkyl_gyrokinetic_species ion = {
...
    .time_rate_multiplier = {
      .type = GKYL_GK_FDOT_MULTIPLIER_MASK_F_FRAC_LOCAL, // Many types, located in gyrokinetic.h
      .f_threshold = 1e-5, // Masks the CFL rate when c0 of f is smaller than 
        // 1e-5 times it's local maximum in configuration space 
      .cellwise_const = true, // must be specified
      .write_diagnostics = true, // writes the multiplier
    },
}

Community Standards

  • Documentation has been updated.
  • My code follows the project's coding guidelines.
  • Changes to layer/zero should have a unit test, e.g., core/zero.

Testing:

  • I added a regression test to test this feature.
  • I added this feature to an existing regression test.
  • I added a unit test to test this feature.
  • Ran make check and unit tests all pass.
  • I ran the code with make valcheck, and it is clean.
  • I ran the code through computer-sanitizer on GPU, and it is clean.
  • I ran a few regression tests to ensure no apparent errors.
  • Tested and works on CPU.
  • Tested and works on multi-CPU.
  • Tested and works on GPU.
  • Tested and works on multi-GPU.

Additional Notes

I'm comparing two simulations here. One is regular, the other with time dilation. It uses GKYL_GK_FDOT_MULTIPLIER_MASK_F_FRAC_LOCAL with f_threshold=1e-3. The first figure is for a maxwellian source R=32 mirror simulation with the pseudo orbit averaging scheme.
image
Subtracting the two lines, we see the percent error (f_time-dilated - f_orig)/f_orig
image

The two lines are nearly identical, but the timestep in the FDP is 3.37466e-10 versus 5.24247e-11. I have found this to be a stable amount of speed up for this problem.

Here are the same two figures, but with the beam source.
image
There are more errors in the beam simulation, but other than right at the mirror throat, they are small.

…s terms. I had a vivid dream that I explained this feature to Ammar and he rejected it first, but liked the way I explained it and wanted to see some tests, so I awoke to make a first pass at this and get the idea out. This is similar to force softening, except it's advective slowdown for everything. The orbit contours should be the same, just slower, so I think this will mitigate the issues we saw before of expander temperature increase due to the force softening. I think this is a really good way to determine the region we want to ignore, by setting the operation scale_fac = min(1.0, dt_omegaH/dt_CFL), so that we are always limited by omegaH. this should screen out the inaccessible regions of phase space automatically for the mirror problem. This can be controlled by setting the omegaH_fac in the input file. I need to figure out how to do this while loop with array methods because this doesn't work for GPUs, but we cannot array divide things so it's a little complicated. We would also need some sort of array_min(a, array) operation, which I need to collaborate with others on.
…ed tests. I should clarify that my dream was explaining that instead of force softening, using the time dialation scheme (dialating the advective term) has a similar property of constraining the time step due to advective restrictions without allowing extra leakage. Insead of force softening, which allows extra stuff to get through the throat, we dialate the time in this region to just made evolution there slower. It preserves the energy contours of the problem while allowing a larger time step. Yes, I did give this detailed description to Ammar in my dream including figures and diagrams.

- Added gkyl_array_min and gkyl_array_min_range functions for element-wise minimum operations.
- Implemented CUDA kernels for minimum operations on arrays.
- Updated tests to include new minimum functionality.
- Removed unused previous step CFL rate logic from species dynamics.
…erations and dg_bin ops. I made a special basis for this operation. Added in the ability for either users to specify the minimum dt, or for the code to minimize dt with the time dialation method based on omega_H. The user can always adjust the omega_H frequency at the input file level by specifying something like cfl_frac for the omegaH mode.
…ate into binary operations; update collisionless app for CFL scaling
… think this means we need to combine this into one big method because we have to allocate everything at initilization. Or at least the initilization needs to initilize everything. Tested the poa regression test and it still works. On to simulations!
…te related calls in tests and applications. Add and verify GPU unit tests for this function
…eshold to determine the minimum dt timestep restriction
…ng to mask CFL. There were some GPU side issues with the array_reduce operation, but it is resolved. Remove all boolean operations for the arrays. Add a skip_cell_advance_inverse operation because using array arithmatic gives FLOP arithmatic errors.
…ask updater instead of the skip cell updater. It's cleaner now, plus I added an essential feature to the array mask to mask things that are some fraction of the maximum f value. I need to test this implementation more thoroughly, but it should be working now. This is just a scripting change in the app, so it should be fine. I updated the reset method to have a more robust decision process of releasing and re-initializing the array_mask so that it can change between the different thresholds and update the threshold
…ier.I tested some regression tests and they are valgrind clean. First results look reasonable, but I can't really tell if it's working correctly as intended. The logic should all be inside the app
…s for each version of time dilation. This is because we only want the allreduce to happen for the local mask. There were also if statements in the advance method, which is a red flag. Format with uncrustify. Must test on GPU and CPU
…ctor (so this is stored inside gk_species). This allows us to dilate electrons relative to ions to have much larger time steps and reduce the scale separation between them.
@Maxwell-Rosen Maxwell-Rosen marked this pull request as ready for review February 16, 2026 18:36
@Maxwell-Rosen Maxwell-Rosen added the enhancement New feature or request label Feb 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DR] Time dilation methods to control CFL in inaccessible regions of phase space

1 participant